Conventionally, there has been an information processing system in which an input is performed by an operation of moving an input device. For example, in a conventional game system, a controller held by a user is provided with a camera, and an image of a marker arranged in the vicinity of a monitor such as a television is captured by the camera. The game system calculates a two-dimensional vector based on the image captured by the camera, and moves an operation target displayed on the monitor based on the two-dimensional vector.
In the method of capturing an image of a marker with a camera, however, a user can move the input device only in a range where the marker is included in the imaging range (field-of-view range) of the camera, and thus the range where the user can move the input device is limited.
Therefore, the present application discloses a storage medium having stored thereon an information processing program, an information processing system, an information processing apparatus, and an information processing execution method, which enable a user to perform an input operation by freely moving an input member.
(1) An example of an information processing apparatus described in the present specification performs predetermined information processing in accordance with an attitude of an input member having an imaging device and an attitude sensor. The information processing apparatus includes a first position calculation section, a second position calculation section, and a processing execution section. The first position calculation section calculates a first position according to the attitude of the input member, which position is on a predetermined straight line or a predetermined plane, by using imaging information based on an image including a predetermined imaging target captured by the imaging device. The second position calculation section calculates a second position according to the attitude of the input member, which position is on a predetermined straight line or a predetermined plane, by using a result of detection of the attitude sensor. When the first position is calculable, the processing execution section performs a scroll process of scrolling an image displayed on the display device, by using a control position based on at least the first position. When the first position is incalculable, the processing execution section performs the scroll process by using a control position based on at least the second position.
The above “input member” may be an input device separated from the information processing apparatus, or may be integrated with the information processing apparatus.
(2) After the scroll process has been started, the processing execution section may start the scroll process by using the control position based on the second position, when the state where the first position is calculable has switched to the state where the first position is incalculable.
(3) The processing execution section may perform, as the scroll process, a process of moving a target image displayed on the display device in accordance with the attitude of the input member. When the state where the first position is calculable has switched to the state where the first position is incalculable, the processing execution section may continue the movement of the target image by using the control position based on the second position.
The above “target image” may be the entirety of an image displayed on the screen of the display device, or an image displayed on a part of the screen. That is, the process of moving the target image may be a process of scrolling the screen, or a process of moving an image of an object displayed on the screen.
(4) When the state where the first position is calculable has switched to the state where the first position is incalculable, the processing execution section may perform the scroll process by using a control position based on the first position before the switching and the second position after the switching.
(5) When the state where the first position is calculable has switched to the state where the first position is incalculable, the processing execution section may specify a correction process for correcting the second position so that the values of the first position and the second position immediately before the switching are equal to each other, and may perform the scroll process by using a control position obtained by performing the correction process on the second position after the switching.
(6) When the state where the first position is calculable has switched to the state where the first position is incalculable, the processing execution section may calculate a difference value by subtracting the second position immediately before the switching from the first position immediately before the switching, and may perform the scroll process by using a control position obtained by adding the difference value to the second position after the switching.
(7) The imaging information for calculating the first position used for calculation of the difference value, and the detection result of the attitude sensor, for calculating the second position used for calculation of the difference value, may be obtained at the same timing by the information processing apparatus.
(8) After the switching from the state where the first position is calculable to the state where the first position is incalculable, if further switching to the state where the first position is calculable has occurred, the processing execution section may perform the scroll process by using a control position which is obtained by correcting the first position by using a value obtained by performing the correction process on the second position immediately after the switching to the state where the first position is calculable.
(9) The information processing apparatus may further include a first pointer processing section. The first pointer processing section displays a pointer image at a position corresponding to the first position, if further switching to the state where the first position is calculable has occurred after the switching from the state where the first position is calculable to the state where the first position is incalculable.
(10) The information processing apparatus may further include a second pointer processing section. The second pointer processing section displays a pointer image at a position corresponding to the control position, if further switching to the state where the first position is calculable has occurred after the switching from the state where the first position is calculable to the state where the first position is incalculable.
(11) The scroll process may be a process of scrolling an image displayed on the display device.
(12) The information processing apparatus may further include a third pointer processing section. The third pointer processing section displays a pointer image for designating a selectable object displayed on the display device, at a position corresponding to the control position, when the scroll process is not performed.
(13) The processing execution section may perform the scroll process while a predetermined condition is satisfied, and may set the amount of correction by the correction process to 0 at the time of starting the scroll process.
(14) Further, the present specification discloses an example of an information processing apparatus which controls an image displayed on a display device, in accordance with an attitude of an input member having an imaging device and an attitude sensor. The information processing apparatus includes a first movement control section, and a second movement control section. The first movement control section configured to move a target image displayed on the display device, in accordance with the attitude of the input member, by using imaging information based on an image including a predetermined imaging target captured by the imaging device. The second movement control section configured to continue the movement of the target image in accordance with the attitude of the input member, by using a result of detection of the attitude sensor, when the predetermined imaging target is outside an imaging range of the imaging device due to a change of the attitude of the input member.
(15) Another example of an information processing apparatus described in the present specification performs predetermined information processing in accordance with an attitude of an input member having an imaging device and an attitude sensor. The information processing apparatus includes a first obtainment section, a second obtainment section, and a processing execution section. The first obtainment section obtains imaging information based on an image including a predetermined imaging target captured by the imaging device. The second obtainment section obtains a result of detection of the attitude sensor. At the time of starting the predetermined information processing, if execution of the information processing based on the image captured by the imaging device is possible, the processing execution section performs the information processing by using the imaging information. When the state where execution of the information processing based on the image captured by the imaging device is possible has switched to the state where execution of such information processing is not possible, the processing execution section performs the information processing with the information to be used being switched from the imaging information to the detection result of the attitude sensor.
(16) When the state where execution of the information processing based on the image captured by the imaging device is not possible has switched to the state where execution of such information processing is possible, the processing execution section may perform the predetermined information processing by using the detection result of the attitude sensor, obtained immediately after the switching, and the imaging information obtained after the switching.
(17) Further, the present specification discloses an example of an information processing apparatus as follows. The information processing apparatus may perform predetermined information processing in accordance with an attitude of an input member having an imaging device and an attitude sensor. The information processing apparatus may include an imaging information obtainment section, an angular velocity information obtainment section, and a processing execution section.
The imaging information obtainment section obtains imaging information based on an image including a predetermined imaging target captured by the imaging device. The angular velocity information obtainment section obtains angular velocity information indicating an angular velocity of the input member as a result of detection of the attitude sensor (e.g., gyro sensor).
When execution of the information processing based on the image captured by the imaging device is possible, the processing execution section performs the predetermined information processing in accordance with a change of the position of the predetermined imaging target in the captured image. Further, when execution of the information processing based on the image captured by the imaging device is not possible, the processing execution section performs the predetermined information processing in accordance with a change indicated by the angular velocity represented by the angular velocity information, instead of the change of the position of the predetermined imaging target in the captured image.
It is noted that the present specification discloses an information processing system capable of performing the same processes as those of the respective components of the information processing apparatuses in the above (1) to (17), and discloses a computer-readable storage medium which causes a computer to perform the same processes as those of the respective components. Furthermore, the present specification discloses an information processing execution method performed in the above (1) to (17).
These and other objects, features, aspects and advantages of the exemplary embodiment will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.